On Local Entropy, Stochastic Control, and Deep Neural Networks
نویسندگان
چکیده
In this letter, we connect some recent papers on smoothing of energy landscapes and scored-based generative models machine learning to classical work in stochastic control. We clarify these connections providing rigorous statements representations which may serve as guidelines for further models.
منابع مشابه
ON LOCAL HUDETZ g-ENTROPY
In this paper, a local approach to the concept of Hudetz $g$-entropy is presented. The introduced concept is stated in terms of Hudetz $g$-entropy. This representation is based on the concept of $g$-ergodic decomposition which is a result of the Choquet's representation Theorem for compact convex metrizable subsets of locally convex spaces.
متن کاملSynchronization control of stochastic delayed neural networks
In this paper, synchronization control of stochastic neural networks with time-varying delays has been considered. A novel control method is given using the Lyapunov functional method and linear matrix inequality (LMI) approach. Several sufficient conditions have been derived to ensure the global asymptotical stability in mean square for the error system, and thus the drive system synchronize w...
متن کاملLocal Deep Neural Networks for Age and Gender Classification
Local deep neural networks have been recently introduced for gender recognition. Although, they achieve very good performance they are very computationally expensive to train. In this work, we introduce a simplified version of local deep neural networks which significantly reduces the training time. Instead of using hundreds of patches per image, as suggested by the original method, we propose ...
متن کاملLearning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
Low-bit deep neural networks (DNNs) become critical for embedded applications due to their low storage requirement and computing efficiency. However, they suffer much from the non-negligible accuracy drop. This paper proposes the stochastic quantization (SQ) algorithm for learning accurate low-bit DNNs. The motivation is due to the following observation. Existing training algorithms approximate...
متن کاملStochastic Pooling for Regularization of Deep Convolutional Neural Networks
We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within the pooling region. The approach is hyper-parameter free and can be combined wi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Control Systems Letters
سال: 2023
ISSN: ['2475-1456']
DOI: https://doi.org/10.1109/lcsys.2022.3189927